# Manual Screen Reader Test Plan

[Return to UniversalMediaController Home](../index.html)

This document is for testing UniversalMediaController directly with a
screen reader and a keyboard. It does not assume you can see the screen.
Every step tells you what key to press and what you should hear.

## Issues found in real JAWS and NVDA testing, and what changed

Real testing sessions found several issues. Issue 1 in particular has now
gone through three rounds of testing and fixes, and it is important to
understand what changed each time and why, since it is still not
considered fully resolved.

**Issue 1: Audio Description, Captions, and Transcript were not
consistently reachable using Tab or form-element navigation**, even
though they were discoverable while browsing.

- *First attempt (did not resolve the problem):* every control's final
  accessible-name and state attributes were set before that control was
  added to the page, and every control was given an explicit
  `tabindex="0"`, while still using `aria-disabled="true"` to mark the
  control unavailable. This did not help.
- *Second attempt (also did not fully resolve the problem):* `aria-disabled`
  was removed entirely, on the theory that JAWS's own quick-navigation
  features filter out elements whose accessibility state reports as
  disabled. Real testing with both JAWS and NVDA this round found the
  problem persisted in JAWS specifically: forward Tab could still skip a
  control described as "unavailable," even without `aria-disabled` present
  -- though Shift+Tab could subsequently reach it. NVDA was more
  consistent but still not ideal.
- *Current fix (a different, more fundamental change):* the underlying
  idea changed, not just the markup. A control whose feature is not
  available is no longer modeled as "a control in an unavailable state"
  in any form. It is modeled as an ordinary, fully enabled button whose
  job is to explain that the feature is not available when you activate
  it -- the same way a normal button's job is to do the thing it's named
  for. Its accessible name says "not available" directly (for example,
  "Captions, not available"), with nothing else marking it as anything
  other than a normal button.

**A standalone diagnostic page now exists specifically for testing this
exact pattern in isolation:** open `Tests/Diagnostic/focus-order-diagnostic.html`
through the same local server (`http://localhost:8000/Tests/Diagnostic/focus-order-diagnostic.html`).
It has exactly four buttons -- Audio Description, Captions, Transcript,
Play -- and nothing else from the framework, so if a reachability problem
still exists, it will show up there without anything else to account for.
**Please test this page first**, before the full demonstrations, and do
not consider Issue 1 resolved unless both the diagnostic page and the
full demonstrations behave correctly.

For the diagnostic page specifically, forward Tab order should be Audio
Description, Captions, Transcript, Play, and Shift+Tab from Play should
reverse that exactly. For the full demonstrations, the same four (plus
the remaining playback controls) should be reachable in the full
accessibility-first order documented below.

**Because two previous fix attempts did not fully resolve this, please
treat this as unverified until your next JAWS and NVDA pass confirms it.**
Specifically check, on both the diagnostic page and a full demonstration:
Tab reaches Audio Description, Captions, and Transcript every single
time, not just sometimes; JAWS's form-field navigation (pressing F)
reaches all three; and JAWS's button navigation (pressing B) reaches all
three. If any of these are still inconsistent, this issue is still open.

**Issue 1b: NVDA reported repetitive speech when entering the
controller.** Landing on the first control (Audio Description) right
after crossing into the controller's group produces two announcements
back to back that both name the media -- the group's own name, then the
first control's name. Both are independently necessary (the first
control for people who navigate directly by form controls and never hear
the group name; the group name so multiple controllers on one page stay
distinguishable), so this could not be fully eliminated. The group's
accessible name was shortened from `Media controls for "Title"` to
`Controls for "Title"` as a partial improvement -- please listen for
whether this reads better, understanding the title will still be spoken
twice in a row on this specific navigation path.

**Issue 2: the first focusable control must identify the media.** This was
already working and remains unchanged: Audio Description, as the first
control, announces the media title as part of its own name (for example,
`Audio Description for "Introduction to Open Door Design", not available`).

**Issue 3: the demonstration media was too loud.** The test tone used in
the video and audio demonstrations has been regenerated at a much lower
volume with a gentle fade in and out, and every demonstration page now
starts at 50 percent volume instead of full volume. If a demonstration
still feels too loud, lower the Volume control -- it responds immediately
and remembers your choice for the rest of that page's session.

**Issue 4: the demonstration pages read like technical documentation
instead of a media experience.** The explanatory paragraphs that used to
sit above and below the controls on each demonstration page have been
removed or shortened. Implementation details now live in
`Docs/Getting Started.md`, `Docs/Architecture.md`, and this document
instead of on the demonstration pages themselves.

## Before you begin

1. Start the local test server. From the repository root, run:

   ```
   python Tools/serve-examples.py
   ```

2. Open Chrome and go to one of these addresses:

   - `http://localhost:8000/Examples/HTML5/video-example.html`
   - `http://localhost:8000/Examples/HTML5/audio-example.html`
   - `http://localhost:8000/Examples/HTML5/multiple-controllers-example.html`

3. On any of these pages, the controller has ten controls, always in this
   order: Audio Description, Captions, Transcript, Play/Pause, Restart,
   Skip Backward, Skip Forward, Playback Speed, Mute, Volume. After Volume
   comes a current time and duration readout, which is not a control -- it
   does not receive focus.

4. None of the controls in this milestone play any audio-described or
   transcript content. Audio Description and Transcript will always
   announce themselves as not available. This is expected, not a failure
   -- see "Test: not-available controls" below for what correct behavior
   sounds like.

## Test: keyboard-only operation

No mouse is needed for anything in this document. If you find yourself
wanting to use a mouse to complete any of the following tests, that is a
failure.

**Expected behavior:** Every control can be reached using Tab and
Shift+Tab, and every control can be operated using Enter, Space, or the
arrow keys. Nothing requires a mouse.

**Failure indicators:** A control cannot be reached with Tab. A control
does not respond to Enter or Space. The Tab key stops moving between
controls (a keyboard trap). Focus leaves the page entirely and Shift+Tab
cannot bring it back.

## Test: Tab and Shift+Tab order

Starting from the top of the page, press Tab repeatedly.

**Expected behavior:** You reach the controls in exactly this order:
Audio Description, Captions, Transcript, Play/Pause, Restart, Skip
Backward, Skip Forward, Playback Speed, Mute, Volume. Pressing Shift+Tab
from Volume reverses this exactly, ending back at Audio Description.

**Failure indicators:** Any control is skipped. The order does not match
the list above. Shift+Tab does not exactly reverse the forward order.
Focus lands on something with no announced name at all.

## Test: form-element navigation

This test simulates a person who jumps directly to form controls without
first passing through a heading or a named group -- a very common way to
navigate a page quickly.

- In JAWS, press **F** to move to the next form field, or open the Forms
  List with **Insert+F5** and select a control directly from the list.
- In NVDA, press **F** in browse mode to move among form fields the same
  way.

**Expected behavior:** However you arrive at the first form control
(Audio Description), it tells you which media item it belongs to before
anything else. You should hear the media title as part of what is
announced, not just "button" or "Audio Description" alone.

**Failure indicators:** The first form control you land on announces no
title. You have to find a heading or a named region first before you
learn what media is involved.

## Test: button navigation

- In JAWS, press **B** to move to the next button.
- In NVDA, press **B** in browse mode to do the same.

**Expected behavior:** Every button you land on has a clear, distinct
name. The first button you reach this way (Audio Description) includes
the media title. Later buttons (Captions, Transcript, Play, Restart, Skip
Backward, Skip Forward, Mute) are short and clear but do not repeat the
title.

**Failure indicators:** Any button announces only as "button" with no
other text. Two different buttons announce identical names with nothing
to tell them apart.

## Test: not-available controls (Audio Description, Transcript, and sometimes Captions)

This test has three parts: whether you can reach the control at all,
what you hear when you activate it, and where focus ends up afterward.
All three matter -- a control that is reachable but silent, or reachable
and vocal but that moves focus somewhere unexpected, both count as
failures.

### Keyboard reachability

Tab to Audio Description. Do this both with plain Tab (pressing Tab
repeatedly from the top of the page) and with your screen reader's
quick-navigation keys (JAWS/NVDA: **F** for form fields, **B** for
buttons).

**Expected behavior:** All three methods reach Audio Description every
time, not just sometimes. The same is true for Transcript, and for
Captions on a page where it has no available track (the audio example).

**Failure indicators:** Any method sometimes skips the control. The
control is reachable one way (for example, by plain Tab) but not another
(for example, by quick-nav). See "Issue 1" above -- this is exactly the
failure that was found in the previous testing round.

### Announcements

With focus on Audio Description, press Enter.

**Expected behavior:** You hear one message, something like "Audio
description is not available for [the media title]." Nothing else is
announced before or after it.

Repeat with Transcript. On the audio example page (which has no caption
file), repeat with Captions as well. Each should produce its own single,
distinct message.

**Failure indicators:** Nothing is announced when you press Enter. The
message repeats itself more than once per press. Two different controls
produce the same message with no way to tell them apart. A dialog or new
window opens.

### Focus behavior

After pressing Enter on a not-available control, press Tab once.

**Expected behavior:** Focus moves to the next control in sequence (for
example, from Audio Description to Captions). It does not return to the
top of the page, jump into the media element itself, or land on the
status message area.

**Failure indicators:** Focus moves anywhere other than the next control
in sequence. Focus is lost entirely (nothing appears to have focus, or
the browser's own address bar receives focus).

## Test: HTML5 video

Open `video-example.html`.

**Expected behavior:** The media title is "Introduction to Open Door
Design." Because this page's video has a caption track built in,
Captions announces itself as available and already on (not
"not available"). Play/Pause, Restart, Skip Backward, Skip Forward,
Playback Speed, Mute, and Volume all work and each announces a single,
short message describing what changed -- for example "Playback started,"
"Playback paused," "Playback restarted," "New position: 10 seconds,"
"Playback speed 1.5 times," "Muted," "Unmuted," "Volume 75 percent,"
"Captions enabled," "Captions disabled." Reaching the end of the media
announces "Playback complete."

**Failure indicators:** Any of the above messages does not play, plays
more than once, or plays continuously (for example, the current time
being read out during ordinary playback -- it should not be).

## Test: HTML5 audio

Open `audio-example.html`.

**Expected behavior:** The media title is "Accessibility Tips Podcast,
Episode 1." Because this audio file has no caption track, Captions
announces itself as not available, the same way Audio Description and
Transcript do. Everything else behaves exactly as in the video test
above.

**Failure indicators:** Same as the video test, plus: Captions announces
as available when it should not be for this page.

## Test: multiple controllers on one page

Open `multiple-controllers-example.html`. This page has two separate
media items, one video and one audio, each with its own set of controls.

**Expected behavior:**

- Each set of controls identifies a different media item. The first one
  you reach is for "Introduction to Open Door Design"; continuing past
  its Volume control and onward reaches a second, separate set of
  controls for "Accessibility Tips Podcast, Episode 1."
- Playing the first item never announces anything about the second item,
  and never changes anything about the second item's Play/Pause state.
- Playing the second item never affects the first item's state either.
- Every announcement you hear while operating one item's controls is
  about that item, never the other one.

**Failure indicators:** Both sets of controls announce the same media
title. Starting playback on one item also starts, pauses, or otherwise
changes the other item. An announcement made while using one item's
controls seems to describe the other item instead.

## Test: JAWS with Chrome (full walkthrough)

1. Launch JAWS. Open Chrome to `video-example.html` (see "Before you
   begin" for the address).
2. Press **Ctrl+Home** to move to the top of the page.
3. Press **Tab** once. You should hear the Audio Description control,
   including the media title, announced as a button, and that it is
   currently not available.
4. Press **Enter**. You should hear one message explaining Audio
   Description is not available for this media item. Focus should still
   be on the same control.
5. Press **Tab** to reach Captions. Press **Enter**. Since this page has
   captions, you should hear either "Captions enabled" or "Captions
   disabled," depending on the current state.
6. Press **Tab** to reach Transcript. Press **Enter**. You should hear
   that Transcript is not available for this media.
7. Press **Tab** to reach Play/Pause. Press **Enter** or **Space**. You
   should hear "Playback started." Press it again; you should hear
   "Playback paused."
8. Press **Tab** to reach Restart. Press **Enter**. You should hear
   "Playback restarted," and the Play/Pause control should read "Play"
   again afterward (Shift+Tab back to it to confirm, if you like).
9. Press **Tab** twice to reach Skip Forward. Press **Enter**. You should
   hear a single message describing the new position, spoken as words
   ("ten seconds"), not digits.
10. Press **Tab** to reach Playback Speed. This is a dropdown; use the
    **Up** and **Down** arrow keys to change it. Each change should
    announce the new speed once, such as "Playback speed 1.5 times."
11. Press **Tab** to reach Mute. Press **Enter**. You should hear "Muted."
    Press it again for "Unmuted."
12. Press **Tab** to reach Volume, a slider. Use the **Left** and **Right**
    arrow keys. Each key press should announce the resulting percentage.

**Failure indicators:** Anything JAWS reads that is not a clear, short
phrase (for example, raw HTML, an unlabeled "button," or a percentage
without the word "percent"). Any repeated or overlapping announcements.
Focus jumping to the top of the page, into the video itself, or into the
status message area at any point during the sequence above.

## Test: NVDA with Chrome (full walkthrough)

Follow the exact same steps as the JAWS walkthrough above. A few NVDA
notes:

- NVDA usually switches into Focus Mode automatically when you land on a
  button, slider, or dropdown, and back into Browse Mode when you move
  away. You do not need to switch modes by hand for the sequence above to
  work.
- If NVDA seems stuck in Browse Mode on a control that should respond to
  Enter or Space, press **NVDA+Space** to force Focus Mode, then retry.

**Expected behavior and failure indicators:** Identical to the JAWS
walkthrough above.

## Test: Vimeo demonstration (experimental -- test this last, and expect rough edges)

Open `Examples/Vimeo/vimeo-example.html` through the same local server.
**This has not been tested with real assistive technology yet.** Treat
this section as a first pass, not a confirmed-working feature -- if
something here doesn't work the way the rest of this document describes,
that is expected and useful information, not necessarily something you
did wrong.

Before testing anything else on this page, first confirm the video
itself loaded: if you hear or see nothing where the video should be,
the demonstration video may no longer be publicly available, and nothing
past that point can be meaningfully tested. If that happens, please note
it rather than assuming the controller itself is broken.

1. Repeat the keyboard-only, Tab/Shift+Tab order, form-element navigation,
   and button navigation tests above on this page. The expected order is
   identical: Audio Description, Captions, Transcript, Play/Pause,
   Restart, Skip Backward, Skip Forward, Playback Speed, Mute, Volume.
2. The media title is retrieved from Vimeo automatically -- no title was
   set on this page directly. Confirm Audio Description (the first
   control) announces a real, meaningful video title, not a placeholder
   like a string of digits. If it announces digits (for example
   "Audio Description for '76979871', not available"), the title did not
   load from Vimeo in time or at all -- note this as a finding.
3. Audio Description and Transcript are expected to always announce as
   not available on this page -- Vimeo does not currently expose an API
   for either. This is expected, not a bug.
4. Captions: if this Vimeo video has captions, confirm the Captions
   control reflects that (on/off, not "not available") and that toggling
   it actually shows or hides captions in the video itself, not just in
   what UMC announces.
5. Playback Speed: change the value. Depending on this specific video and
   Vimeo account's settings, this may succeed (announcing the new speed)
   or may not be available at all (the control should say so, not fail
   silently). Either outcome is possible and expected -- what matters is
   that one of those two things happens clearly, not a silent failure or
   a console error.
6. Play, Pause, Restart, Skip Backward, Skip Forward, Mute, and Volume
   should all behave the same as the HTML5 examples.

**Failure indicators specific to this page:** A JavaScript error appears
in the browser console. The page becomes unresponsive. Any control
freezes or stops responding after an action. Success is announced for an
action that Vimeo silently rejected (for example, hearing "Playback speed
1.5 times" when the video did not actually change speed).

Since Vimeo's own native player controls are still present inside the
video itself for this milestone (see `Docs/Limitations.md`, "Vimeo
Adapter Milestone 1"), you may notice two ways to control the same video.
This is a known, documented tradeoff for this milestone, not something to
report as a new finding -- please do note, however, whether having both
sets of controls created any confusing screen reader behavior (for
example, whether tabbing ever unexpectedly moved focus into the Vimeo
iframe's own controls instead of staying among UniversalMediaController's
controls outside it).